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(g) Method and apparatus for Improving motion compensation in digital video coding. 



@ A novel method and apparatus for use in 
digital video compression provides improved 
bloclc-based motion compensation using the 
global motion of a video frame. A video frame 
comprising a plurality of bloclcs may be conn- 
pressed for transmission using blocl< motion 
vectors. Motion vectors are generated by blocic 
matching a block to be coded with a block 
within a reference frame and determining the 
displacement therebetween. To effect block 
matching, motion compensation techniques de- 
fine a search window within one or more refer- 
ence frames within which a displaced block will 
be found. According to the present invention, 
the locatton of the search window within a 
reference frame is defined using the global 
motion of the firame. 
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Fiatd of the Invention 

The present invention relates to the field of digital 
video data compression, and in particular video data 
compression using block-based motion compensa- s 
tion. 

Background of the Invention 

Block-based motion compensation techniques 10 
Including either motion interpolation or motion predic- 
tion are used in digital image sequence coding to re- 
duce the amount of data that must be transmitted in 
digital video systems. Video sequences typically ex- 
hibit a substantial amount of repetitive image informa- is 
tion between consecutive frames. By compensating 
for the motion of objects within an image frame, a 
high compression of the video data can be achieved. 

Block-based motion compensation techniques 
initially divide a video frame into several blocks of pic- 20 
ture elements (pels). These techniques employ mo- 
tion vectors to indicate the movement of these blocks 
between successive frames. It is assumed that each 
pel within a block has the same transtational motion. 
These motion vectors, along with error signals, may 25 
be transmitted to a receiver. The receiver may then 
regenerate the video blocks using the motk>n vectors 
to locate the corresponding blocks from a previously 
transmitted frame. The error signals are used to ac- 
count for pel differences within each block between 30 
successive frames. 

Consider, for example, the inrtage of a football in 
a football game video sequence. The football image 
remains substantially identical from frame to frame, 
but its location within the video frame changes. If the 35 
motion vector of the football is known, then the Image 
of the football can be reconstructed in a new video 
frame using the football image data from the previous 
frame. As a consequence, instead of transmitting re- 
petitive image data for each new frame, motion vec- 40 
tors may be transmitted. An error signal is also trans- 
mitted to account for other differences, for example, 
a change In orientatbn of the football. 

Motion compensation methods employ a process 
known as motion estimation to generate the motion 45 
vectors. In motion estimation, a motion vector for a 
particular block within a current frame is determined 
by matching the image data within the block to the im- 
age data in a displaced block within a reference 
frame. The motion vector represents the difference in so 
position between the block from the current frame 
and the displaced block from the reference frame. 
The reference frame may be a previous frame, as in 
motion compensation prediction techniques, or a pre- 
vious or future frame, as in motion compensatton in- ss 
terpolation techniques. 

To effect such block matching, the entire refer- 
ence frame is not ordinarily searched for the dis- 
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placed block, as such a search can consume consid- 
erable computation power and time. Instead, a search 
window is defined within the reference frame. 

In motion estimation, a search window may be 
defined in terms of its size in pels and its location 
within the reference frame. In determining search 
window location. It is known in the art to center the 
search window on the position of the block to be 
matched. The method assumes that an image within 
a video block is most likely to be found In the vicinity 
of its location in a previous or subsequent frame. The 
search window size is then optimally chosen to be as 
large as the sum of the block size and the maximum 
probable block movement in any direction. For exam- 
ple, for a 15 X 1 5 pel block capable of moving up to 5 
pels in any direction between frames, the maximum 
search window size would be 25 X 25 pels. 

Referring again to the analogy of a football within 
a football game video sequence, the football may, for 
discussion purposes, represent a block of video data. 
To find a displaced football In a reference frame, a 
motion estimation method would Initially define a 
search window within which it would expect to find the 
football image in the reference frame. The search 
window Is then scanned for the image within the win- 
dow most resembling the football. Once the football 
image is found, a motion vector may be calculated for 
the football describing its movement between the ref- 
erence frame and the present frame. 

Under such methods of locating the search win- 
dow, however, the search window size used for block 
matching limits the range of motion that may be conv 
pensated for. In the 25 x 25 pel search window de- 
fined above, block motion of more than 6 pels cannot 
be successfully compensated. If the motion estima- 
tion method fails to find the proper displaced block, a 
large error signal will have to be transmitted, and the 
advantages of motion compensation will be greatly 
reduced. 

To increase the range of motion which may be 
compensated for, It is well known that the search win- 
dow size may be enlarged in order to account for 
greater image movement. The use of larger search 
windows, however, consumes more computation time 
and power because there are mors candidate dis- 
placed blocks to evaluate to effect block matching. 
For example, If an exhaustive search is used to effect 
block matching, an Increase in size of a 25 x 25 search 
window by a single pel In each direction will increase 
the number of candidate displaced blocks by 27^ 
252=104 blocks. 

Several attempts have been made to increase 
the range of motion which may be compensated for 
without drastically increasing the computational load. 
It is well known In the art to employ search algorithms 
which are faster than an exhaustive search. These al- 
gorithms include the hierarchical search, the logarith- 
mic search, and conjugate direction, search. Because 
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motion estimation techniques using such algorithms 
greatly reduce the number of calculations required for 
block matching, they may employ larger search win- 
dows than an exhaustive search routine. 

The faster search techniques, however, are not s 
guaranteed to find the proper displaced block. In 
other words, these algorithms may not find the best 
match for a particular block to be coded, which leads 
to coding inefficiency. Furthermore, such search, 
techniques require complex Image data manipulation io 
when compared with the exhaustive search. The 
complex image data manipulation may also computa- 
tionally load the system. 



Summary of the Invention 



IS 



The present invention provides a method of mo- 
tion estimation operable to enable compensation for 
a large range of motion using a novel method of locat- 
ing the search window. Large amounts of motion in 20 
video are often due to a combination of object motion 
and global motion caused by, for example, camera 
motion. According to the present invention, the 
search window is placed in a location that takes into 
account the global motion of the entire video frame, 25 
thus enabling compensation for greater amounts of 
motion when a component of the motion is caused by 
global motion. 

For example, again consider the image of a mov- 
ing football in a football game video sequence. The 30 
motion of the football is often due to camera motion 
as well as object motion, which in the process of block 
matching may cause the football image to fall outside 
a reasonably sized search window. If, however, the 
search window is placed in a location in a reference 3S 
frame that takes into account the camera motion, 
then the search window would more likely contain the 
football image, thus facilitating successful compen- 
sation. 

Accordingly, the method of the present invention 40 
generates a global motion vector from time to time 
and transmits this data to a motion estimation device. 
To perform block matching, the motion estimation de- 
vice uses the global motion vector to define a search 
window in which a displaced block may be found with- 4S 
in one or more reference frames. The motion estima- 
tion device then identifies the displaced block within 
the defined search window. The method of the pres- 
ent invention is compatible with both motion compen- 
sation prediction and motion compensation Interpola- so 
tion techniques. 

The present invention may be utilized with either 
an exhaustive search in block matching, or it may also 
be used with faster search techniques. 



Brief Description of the Drawings 

FIG. 1 shows a block diagram of a video coder cir- 
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cuit employing a novel block-based motion com- 
pensation method and apparatus according to 
one embodiment of the present invention; 
FIGs. 2A and 2B show a video image frame div- 
ided into blocks of pels in accordance with stan- 
dard methods of video data coding techniques; 
FIG 3A shows a functional flow diagram of the 
steps performed by a motion estimation device 
according to the present Invention; 
FIG. 3B illustrates how a global motion vector 
may be utilized in accordance with the present in- 
vention to define a search window within a refer- 
ence frame to more effectively achieve block 
matching according to the present invention; and 
FIG. 4 shows a functional block diagram of a glo- 
bal motion estimator operable to generate global 
motion vector data for use in the methods and ap- 
' paratus of the present inventksn. 

Detailed Description 

The present invention provides an improvement 
to current motion estimation techniques used in 
block-based motion compensation by using the glo- 
bal motion, caused by, for example, camera motion, 
to assist in defining a search window in the process 
of block matching. A novel video coder circuit' 1 00 em- 
ploying motion estimation techniques according to 
the present invention is described below in connec- 
tion with FIG. 1. 

FIG. 1 shows, in block diagram form, v\6ho coder 
circuit 100 for encoding a sequence of image frames 
represented in digital form using block-based motion 
compensation. The block diagram is shown in simpli- 
fied form for purposes of clarity. To this end, various 
aspects of digital circuitry are not represented jn the 
diagram and should be considered as inherenf to the 
system. For example, a timing and sequence control- 
ler for the various functional blocks, as well as Imbed- 
ded storage elements such as buffers and memories 
are not shown. The details pertaining to the use of 
such digital circuitry in a system according to the 
present invention are well known or will be readily ap- 
parent to one of ordinary skill in the art. 

The video coder circuit 100 receives at its input 

101 video data corresponding to temporal frames of 
video. A frame of video comprises a plurality of pic- 
ture elements or pels. The digital representation of 
the video frame as it appears at the input 1 01 of video 
coder circuit 100 comprises pel vkieo data for each 
pel within the frame. The pel video data typically com- 
prises chrominance and luminance values. A video 
frame may be further subdivided into blocks of pels, 
and frames are processed by the video coder circuit 
100 on a block by block basis. 

The video coder circuit 1 00 produces at its output 

102 a compressed and encoded digital signal in a bit 
stream for subsequent storage or transmission over 
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a transmission medium, not shown. The compressed 
and encoded digital signal may be suitably coded ac- 
cording to the MPEG1, MPEG2 or any other video 
data transmission standard that is designed specifi- 
cally to work with data compression techniques in- 5 
eluding block-based motion compensation prediction 
and interpolation. See, for example, ANSI Standard 
ISO-IEC DIS 11172, "Coding of Moving Pictures and 
Associated Audio for Digital Storage Media For Up to 
About 1 .5 Mbit/sec/ io 

The video coder circuit 100 illustrated in FIG. 1 
compresses and encodes data employing motion 
compensation prediction techniques. In processing 
the blocks of a video frame, the circuit 100 may either 
encode a particular video data block directly or en- is 
code the block using motion compensation predic- 
tion. Video blocks that cannot be successfully com- 
pensated are encoded directly. 

The functional elements in the video coder circuit 
1 00 are interconnected as follows. A video frame stor- 20 
age unit 103 is operably connected to transfer both 
vkieo data and position data corresponding to a new 
block of video to be coded ("new block") to a motion 
estimator 104, and to transfer the same new block 
video data to both a subtraction node 105 and a cod- 25 
ing device 107. The video frame storage unit 103 in- 
cludes a digital memory or buffer which stores both 
video data corresponding to one or more frames and 
is operable to generate and store block position data 
for the video blocks within each stored frame. 30 

The motion estimator 104 is operably connected 
to receive block video data from the video frame stor- 
age unit 1 03 and global motion data from a global mo- 
tion estimator 106. The motion estimator 104 com- 
prises a processing means such as a microprocessor 3S 
and related memory circuitry which is operable both 
to effect block matching using global motion data and 
to generate motk>n vectors. The motion estimator 1 04 
further includes motion compensation means for pro- 
viding displaced block data to the subtraction node 40 
105. According to one embodiment of the invention, 
the processing means within the motion estimator 
104 is programmed to execute the block matching 
and motion vector generation consistent with the flow 
chart 300 described below in connection with FIG. 3A. 45 
The motion estimator 104 is further connected to 
transfer block video data to the subtraction node 105 
and to provide motion vector data to both a coding de- 
vice 107 and the global nrwtion estimator 106. 

The motion estimator 1 04 is further connected to so 
a reference frame storage buffer 108 and a new 
frame storage buffer 1 09. The new frame storage buf- 
fer 109 is connected between an additton node 110 
and the reference frame storage buffer 108. Both the 
new frame storage buffer 109 and the reference 55 
frame storage buffer 108 have sufficient storage 
space to hold the video data corresponding to at least 
one video frame. 



The subtraction node 105 is operably connected 
to receive new block video data from the video frame 
storage unit 103 and to receive displaced block video 
data from the motion estimator 104. The subtraction 
node 105 is further connected to the coding device 
1 07. The subtraction node 1 05 comprises a data stor- 
age means, such as a buffer and means for generat- 
ing an error signal representing the difference in pel 
luminance and chrominance between two video data 
blocks received from the motion estimator 104 and 
the video frame storage unit 103. 

The coding device 107 is operably connected to 
receive the error signal from the subtraction node 
105, motion vector data from the motion estimator 
1 04 and block video data from the video frame stor- 
age unit 103. The coding device 107 is further con- 
nected to provide an error signal to the addition node 
110, and includes the output 102 for transmitting cod- 
ed video data in the form of a bit stream. The coding 
device 107 comprises a means for encoding and de- 
coding video data using the MPEG1, MPEG2 or other 
coding standard for transmitting a coded bit stream. 
Devices capable of coding in these standards are 
known in the art. For more information on such coding 
methods and devices, see ANSI Standard ISO-iEC, 
above, incorporated by reference herein. 

The global motion estimator 106 is connected to 
receive motion vectors from the motion estimator 104 
and to provide the global motion vector to motion es- 
timator 104. The global motion estimator comprises 
the processing means and related circuitry operable 
to store a plurality of motion vectore and to compute 
therefrom a global motk>n vector that is representa- 
tive of a common motion component within an entire 
video frame. Although the global motion estimator 
106 is illustrated as aseparate element, the functions 
performed by the global motion estimator processing 
means may be performed by either a dedicated mi- 
croprocessor or a microprocessor included within the 
system to perform other control and operational func- 
tions. To achieve its desired functions, the global mo- 
tion estimator 106 may suitably be programmed con- 
sistently with the flow chart 400 discussed below in 
connection with FIG. 4. 

The addition node 110 is connected to receive 
block video data from the motion estimator 104 and 
error signal from the coding device 107. The addition 
node 110 comprises means for combining a block of 
vkJeo data with an error signal to produce a compen- 
sated block of video data. The addition node provides 
the compensated block to new frame memory buffer 
109. 

The video coder circuit 100 described above op- 
erates in the following manner to process and encode 
a new video frame, one block at a time, using block- 
based motion compensation with global motion esti- 
mation. A video frame to be coded is f irat divided into 
blocks. Each block is then encoded either directly or 
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using motion compensation prediction until the entire 
frame is encoded. The decision of whether or not to 
encode using motion compensation is made by the 
control circuitry using predetermined criteria, typical- 
ly dependent on the size of the error signal produced 
during motion compensation. The next frame is there- 
after processed in the same manner. 

It should be noted that the various elements dis- 
cussed below exchange data using digital signals, the 
generation of which is known to those of ordinary skill. 
For purposes of clarity, it will be understood that when 
any element Is said to provide or receive data, for ex- 
ample, representing video blocks or motion vectors, 
to or from another element, the element is actually 
providing or receiving digital signals containing the 
data. 

In order to process a video frame one block at a 
time, the control circuitry directs that each frame, pri- 
or to processing, be divided into blocks of video data, 
DB|,t. where i is the block number within a frame cor- 
responding to a time t. Initially, video data defining a 
video frame Ft corresponding to a time t in a video se- 
quence is stored in a buffer or other storage means 
within the video frame storage unit 1 03. The time t ac- 
tually represents a sequential frame number such 
that, for example, Ft-i is the frame immediately pre- 
ceding Ft, and Ft*i is the frame Immediately subse- 
quent to Ft- The control circuitry directs that each 
frame be divided into a number of blocks per row m. 
and a number of blocks per column n, so that the 
frame has total number of blocks, NB, or in other 
words, m X n = NB blocks. The control circuitry may 
be user controllable to select the number of blocks per 
row m and the number of blocks per column n. Once 
the number of blocks is determined, the video frame 
storage unit 103 then obtains and stores block posi- 
tion values BPi,t for each block within one video frame. 

For the present embodiment of the invention, the 
block position value may suitably be defined as the 
pel position value of the pel in the upper left hand cor- 
ner of each block. The pel position value constitutes 
the position within the two-dimensional array of pels 
comprising the video frame. The pel position may 
suitably be provided by the control circuitry. The vid- 
eo frame storage unit 103 may determine the block 
position for each block i, BPj,t= (x,y). using any appro- 
priate method. 

For example, FIG. 2A illustrates the frame Ft 201 , 
which is divided into 16 rows and 16 columns of 
blocks. Block 202 represents data block DBi9.t in 
frame 201 wherein the first block 203 is DBo.t and the 
block i-values are counted from left to right and top to 
bottom. Block 202, as shown in FIG. 2B comprises a 
15X15 array of pels, one of which is exemplif ied by 
upper left corner pel 204. The pel position value for 
pel 204 is (45.15), corresponding to its x,y coordin- 
ates. These coordinates are determined as follows. 
Pel 204 is the upper left hand pel of block 202. Block 



202 is the 19th block in frame Ft 201. and it is in the 
fourth column and second row of frame Ft 201 . X-pels 
0-14 are in blocks in the same column as block 203, 
x-pels 15-29 are in blocks in the second column, x- 

5 pels 30-44 are in blocks in the third column, and x- 
pels 45-59 are In blocks in the fourth column, which 
Is the column block 202 is in. Thus, the left most pels 
of block 202 will have an x-coordinate of 45. Li kewise, 
y-pels 0-14 are in blocks in the same column of block 

10 203, and y-pels 15-29 are in blocks in the next col- 
umn, which is the column block 202 is In. Thus, the up- 
per nrrast pels of block 202 will have a y-coordinate of 
15. The block position value of block 202, as dis- 
cussed above, is the pel position value for its upper 

IS left hand corner pel 204. Therefore, BPtg.t = (45.15). 

Once the data blocks and their respective block 
position values are defined, the control circuitry di- 
rects that each block DB,.t be processed, block by 
block. Returning to FIG. i, the control circuitry in- 

20 structs the video frame storage unit 103 to provide a 
new block of video data, DB|,t and the corresponding 
block position data. BPi,t to the motion estimator 104. 
The new block video data DB|,t is further transmitted 
to the subtraction node 105 and stored in a buffer 

25 therein. 

The motion estimator 104 then effects .block 
matching in order to match DB|,t with a data block, 
MBref. from the video frame stored in the reference 
frame storage buffer 108. The storage buffer 108 

30 contains at least one reference video frame. Because 
the block is to be encoded using motion compensa- 
tion prediction in the present embodiment, the refer- 
ence frame comprises the previous video frame. Ft- 
1. In motion compensation prediction, the location of 

35 the displaced block represents the location of the vid- 
eo image defined by the new block within the previ- 
ous frame. 

If, however, the new block is to be encoded using 
motion compensation interpolation techniques, the 
40 motion estimator 104 locates MBref within a plurality 
of reference frames stored in the reference frame 
storage buffer 109. The plurality of reference frames 
may correspond to both previous and/or future video 
frames depending on the chosen motion compensa- 
45 tion technique. In such cases, the motion estimator 
104 performs the block matching step for the plurality 
of reference frames. 

The motion estimator 104 is operable to effect 
block matching for each reference frame in the follow- 
so ing manner. The motion estimator 1 04 first defines a 
search window within the reference frame in which 
the displaced block may be expected to be found. Ac- 
cording to the present invention, the motion estimator 
104 uses a global motion vector in defining the loca- 
55 tion of the search window. The global motion vector 
is representative of motion common to all the blocks 
within the video frame Ft, and is from time to time pro- 
vkied by the global motion estimator 106 as dis- 
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cussed In detail further below. The defined search 
window is thereafter searched until the displaced 
block MBnrffor the new block DB,,t is found. 

Once a displaced block MBr«f is found, the motion 
estimator 104 determines the motion vector MV|.dt 
corresponding to the new block. The value dt Indi- 
cates the frame number difference between Ft and 
the reference frame. For motion prediction, the refer- 
ence frame will typically be the previous frame, and 
therefore dt = 1. The motion vector MV|^ comprises 
a two-dimensional value (dx,dy) representing the hor- 
izontal and vertical displacement between the new 
block DB,,t and the displaced block MB,^ The two-di- 
mensional motion vector value (dx,dy) may suitably 
be determined by calculating the difference between 
the new block position value BP,.t and the pet position 
value of the pel in the upper left corner of the dis- 
placed block MBrtrf. For example, referring to Fig. 2, 
the block 202 (DBigJ has a block position value BP1 9,t 
= (45.15). If it is further assumed that a displaced 
block has been identified as corresponding to block 
202 and that the pet position value of the upper left 
hand corner pel of the displaced block MBrof is 
(50,21), then the motion vector MS/iqa = (dx,dy) = 
(+5.+6). 

Af unctionai block diagram of an exemplary block 
matching and motion vector generating method which 
may be employed by the motion estimator 104 to ach- 
ieve the foregoing is discussed further below in con- 
nection with FIG. 3A. 

The displaced block video data MB^f in the mo- 
tion compensator 104 is then transmitted to the sub- 
traction node 105, The displaced block video data is 
also transmitted to the addition node 11 0 for purposes 
discussed further below. In the subtraction node 105, 
the displaced block video data MBref. comprising both 
luminance and chrominance data, is subtracted on a 
pel by pet basis from the new block video data DB|.t 
received and stored from the video frame storage unit 
103. The resulting error signal ERROR represents 
the difference between the displaced block MBr«f and 
the new block DB^. The signal ERROR is then provid- 
ed to the coding device 107. 

The motion estimator 104 provides the motion 
vector MV,.dt to the coding device 107. The motion 
vector is also provided to the global motion estimator 
1 06 for purposes discussed further below. The coding 
device 107 comprises digital circuitry, which may in- 
elude a microprocessor, that is capable of producing 
digital signals according to the MPEG1, MPEG2 or 
other standard to the video coder output 102. 

The coding device 107 also receives the new 
block video data DB,4 from the video frame storage 
buffer 103. The control circuitry then determines 
whether the motion compensated version of the 
block, in other words, the motion vector and the signal 
ERROR, should be encoded and transmitted. If not, 
the control circuitry instructs the coding device 107 



to encode the new block video data DB|,f It however, 
the motion compensated block is to be transmitted, 
the control circuitry instructs the coding device 107 
to encode for transmission the motion vector MV|^ 

5 and the signal ERROR. The encoded data is then pro- 
vided to the circuit output 102 for transmission to one 
or more receiving devices, not shown. In either case, 
the encoded data is transmitted in a bit stream along 
with appropriate signals to indicate whether the mo- 

10 tlon compensated version of the block has been 
transmitted. 

After transmission, the new block DB14 stored 
In order to build the current frame in new frame stor- 
age buffer 109. If the motion compensated version of 
15 the block was transmitted, the addition node 110 wilt 
receive a decoded version of the coded error signal 
from the coding device 107 and will add the error sig- 
nal to MBrof which is received from the motion estima- 
tor 104. The resulting block, called a compensation 
20 block, closely resembles the new block DBi.t. The 
compensation block is placed Inside the new frame 
buffer 109 In a position corresponding to the new 
block position value, BP, t. 

If, however, the motion compensated versfon of 
25 the block was not transmitted, the new block video 
data DB,^t may be placed directly into the new frame 
buffer 1 09 from the motion estimator 1 04. In this man- 
ner, the frame F, is built within the new frame storage 
buffer 109 which will eventually be placed in the ref- 
30 erence frame storage buffer 1 08. 

Concurrent with the operation of the coding de- 
vice 1 07 and the addition node 1 1 0, the global motion 
estimator 106 receives the nrotion vector MV,,df The 
motion vector MN/i.^t is received into a buffer, stack or 
35 memory within the global motion estimator 106. 

The process performed by video coder circuit 
100, as outlined above, is repeated for each block 
DB|,, of each frame until an entire video frame (i=NB) 
has been encoded by the video coder circuit 100. 
40 Once the entire frame Ft has been processed, the 
control circuitry wilt cause the new frame buffer 109 
to transfer the video data corresponding to the frame 
Ft to the reference frame storage buffer 108 in order 
to process the next frame corresponding to the time 
45 t+1. 

Furthermore, before the next frame is processed, 
the global motion estimator 106 will generate and 
transmit a new global motion vector to the motion es- 
timator 104. The global motion estimator 106 gener- 
50 ates the global motion vector using the motion vec- 
tors stored during the processing of Ff The global mo- 
tion vector approximates the component of motion 
common to every block in a frame of a video se- 
quence which may be due to camera motion. The mo- 
ss tlon estimator 1 04 receives and stores the global mo- 
tion vector GMV prior to receiving the new video data 
blockforthe first block ofthenextframe (DB0.H1). The 
global motion estimator may be programmed to exe- 
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cute the above functions as discussed below in con- 
nection with FIG. 4 

It will be understood that the video coder circuit 
100 described in connection with FIG. 1 is exemplary. 
The global motion estimation method and apparatus 
of the present invention may be utilized in any block- 
based video motion compensation device that per- 
forms block matching to compute motion vectors be- 
tween a block to be coded and a displaced block with- 
in one or more reference frames, by first defining a 
search window in the reference frames. For example, 
U.S. Pat. No. 5,151.784 Incorporated herein by refer- 
ence Illustrates a video coder circuit incorporating 
such a motion compensation device that is operable 
to employ t>oth motion compensation prediction and 
motion compensation interpolation for coding block 
based video data. The method according to the pres- 
ent invention may readily be adapted for use therein. 

FIG. 3A illustrates a functional flow chart for a 
motion compensation device that employs a global 
motion vector to effect block matching according to 
the present invention. The flow chart may suitably be 
executed by the motion estimator 104 In FIG. 1. The 
functionality of the flow chart, however, is designed 
to be compatible with alternative video coder circuits 
employing both motion compensation prediction and 
motion compensation interpolation. The processing 
means and memory means (processor) within the 
motion estimator execute a program performing the 
steps in FIG. 3A to effect block matching and motion 
vector generation. 

The processorfirst receives new block video data 
DB|,t for a block i within a frame Ft in step 302. Next, 
in step 310, the search window Aref within the refer- 
ence frame is defined. A^f is video data representing 
the portion of the reference frame within which a dis- 
placed block MBref for DB|,t may be located. The loca- 
tion of Araf within a the reference frame is defined by 
first centering Araf on the new block position BPi,t, and 
then displacing Ar«f by an amount defined by the stor- 
ed global motion vector GM. 

It should be noted that the global motion vector 
GM is normalized to represent the global motion be- 
tween the current frame and the immediately preced- 
ing frame, F^. i. As a consequence, if the reference 
frame is not the immediately preceding frame, the 
global vector should be scaled accordingly. Scaling 
may suitably be linear. For example, if the reference 
frame corresponds to a time t-2, then the global mo- 
tion vector should be doubled before determining the 
location of Arer. Likewise, in motion compensatton in- 
terpolation, the reference frame may be a future 
frame, in which case the global motion vector should 
be inverted. By storing and subsequently scaling a 
normalized global motion vector,, the processor may 
suitably apply the global motion vector to block 
matching operations in both interpolated and predict- 
ed modes. 



The size of the search window A^t should prefer- 
ably be large enough to ensure inclusion of the dis- 
placed block, but small enough so that the block 
matching function does not unduly load the process- 

5 ing means of the motion estimator. The size of search 
window Aref is a predetermined number which takes 
into account the maximum object motion between 
video frames and may vary depending upon the ap- 
plication, as well as other factors. In the context of a 

10 particular application, the determination of a proper 
size will be apparent to one skilled in the art. 

FIG. 3B further illustrates the novel method of 
defining the search window Aref as described in con- 
nection with step 310. In FIG. 3B, the given block 354 

15 from the frame Ft, effects block matching to identify 
displaced block 351 from the reference frame, which 
may suitably be the previous frame. 

The image in a data block OB|,t is assumed to 
have moved from the position of block 351 in the ref- 

20 erence frame to the position of DB^ shown by the 
block 354. The movement of the image in DB,,t is due 
in substantial part to camera movement or some other 
source of global motion. The reference frame 352 has 
the same dimensions as the video frame to be coded 

25 from which DBj^ originated. 

The search window A^f is located by first center- 
ing the search window on the position of the block 
position for the block 354, as illustrated by centered 
window 356. The search window is then displaced by 

30 the global motion vector 355, as illustrated by dis- 
placed window 353. The global motion vector 355 is 
representative of the component of motion, which 
may be caused, for example, by camera movement, 
that Is common to all blocks within a frame. The global 

35 motion vector 355 may suitably be provided from time 
to time by the methods described below in connection 
with FIG. 4. 

The use of global motion in defining the search 
window location provides a clear improvement over 

40 the prior art. Referring to FIG. 3B, the prior art method 
for locating the search window consists essentially of 
centering the search window around the position of 
the new block 354, as illustrated by centered window 
356. See, for example, Jain, et al. "Displacement 

45 Measurement and its Application in Interf rame Image 
Coding," 29 IEEE Transactions on Communications 
pp. 1799^1808 (1981) incorporated herein by refer- 
ence. It can be seen that the centered search window 
356 does not Include the displaced block 351. if the 

50 displaced block is not found, the motion estimator 
cannot effectively aid video compression. 

The centered search window 356 could, of 
course, be enlarged to encompass displaced block 
351. As discussed above, however, enlarging the 

55 search window can greatly increase the number of 
calculations required for block matching. Further- 
more, because many video sequences have little or 
no camera motion, it would be wasteful to enlarge the 
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search window when a smaller search window is ade- 
quate In still sequences. By using global motion, the 
search window can remain a reasonable size yet still 
encompass the displaced block In large camera 
movement sequences. Consequently, because the 5 
use of global motion allows for a larger range of mo- 
tion when camera or global motion is present, it pro- 
vides a substantial improvement to motion compen- 
sation techniques. 

Returning to FIG. 3A, once the size and location io 
of the search window A^f is chosen, the motion esti- 
mator Identifies a displaced block MB„f within the 
search window using predetermined criteria in steps 
312, 314 and 316. The identification of a displaced 
block within a reference frame is well known in the art. is 
See. for example, Jain, et al. above, at 1799-1808. 
The displaced block is a block of data within the 
search window having the least image difference 
when compared to the DB,.t. In step 312. the block 
DB|,t is compared with several candidate displaced 20 
blocks in the search window in order to determine the 
candidate block having the least image difference. In 
step 314, the data for the candidate blocks is retrieved 
as needed from a means for storing the reference 
frame, for example, the reference frame storage buf- 25 
far 108 of FIG. 1. The image difference may suitably 
constitute the mean-absolute-difference or the 
mean-squared-enror between the total pel luminance 
value for the new block DB^ and the total pel lumi- 
nance value for each candidate displaced block. For 30 
more information on such block matching techniques, 
see, for example, Jain, et al, above, at 1800; Sriniva- 
san, et al, "Predictive Coding Based on Efficient Mo- 
tion Estimation." Proc. Infl Conf. Communications 
pp. 521-26 (Amsterdam. May 14-1 7. 1 984). 35 

The choice of candidate blocks within the search 
window for which such image comparisons are made 
In step 312 is dictated by a predetermined search 
methodology. The search methodology may suitably 
be an exhaustive search, which compares every de- 40 
finable block within the search window with the new 
block. The exhaustive search is often preferable be- 
cause it is guaranteed to find the block in the search 
area having the minimum error. The search method- 
ology may alternatively be a logarithmic search, a hi- 45 
erarchical search or other search method which is 
known in the art. For more information on these alter- 
native search methodologies, see, for example, Jain, 
et al, above; Srinivasan, et al. above; and Bierling, 
"Displacement Estimation by Hierarchical Block- so 
matching." Proc. Visual Communications & Image 
Processing '88. (SPIE, November 9-11. 1988). 

Once the displaced block MBrer is identified, the 
motion estimator 1 04 derives the motion vector MV|,di 
for the new block DBi^ in step 31 8. The motion vector 55 
is defined as the displacement between the new 
block position BP,,t and the pel position of the upper 
left hand corner pel, or the first pel. of the displaced 



block, MBref. The displaced block is thereafter pro- 
vided in step 320 to other elements in the coding cir- 
cuit, for example, the subtraction node 105 and addi- 
tion node 110 as Illustrated in FIG. 1 above. In step 
322, the motion vector MV^ is provided to the global 
motion estimator and other elements in the coding 
circuit, for example, the coding device from FIG. 1. 

While motion compensation prediction employs 
only one reference frame, motton compensation in- 
terpolation performs block matching using more than 
one reference frame. As a consequence, in step 324. 
the processor must determine if there is another ref- 
erence frame with which to perform block matching. 
If in step 324 the processor detemiines that another 
block matching step must be performed, the proces- 
sor executes step 326. In step 326. the processor in- 
structs the reference frame storage means to make 
the next reference frame available for block match- 
ing. The processor then returns to step 310 to define 
a search window within the new reference frame. If, 
however, the answer to step 324 is no, the processor 
executes step 328. 

In step 328, the processor determines whether 
there are more new data blocks within the current 
frame to be compensated. This information may suit- 
ably be provided by the control circuitry. If no, the 
processor retrieves a new global motion vector GMV 
from the global motion estimator in step 330. The glo- 
bal motion estimator may suitably operate according 
to the method discussed below in connection with 
FIG. 4. In step 332. the new global motion vector is 
stored as GM for use within the motion estimator 104 
for processing the next vWeo frame, which in motion 
compensation prediction would typically be Ft+i. The 
motion estimator 1 04 thereafter executes step 302. If, 
however, the answer in step 328 is yes, the motton es- 
timator 104 proceeds directly to step 302 to receive 
further blocks from the current frame. 

FIG. 4 shows an exemplary functional flow dia- 
gram of the novel global motion estimation program 
according to the present invention. The program Is 
executed by processing means within a global motion 
estimator such as the global motion estimator 106 
that is discussed above in connection with FIG. 1 . The 
global motion estimator may suitably be a digital mi- 
croprocessor circuit with associated memory and buf- 
fer configurations capable of executing the software 
as well as receiving and transmitting the appropriate 
data from and to other components. Such circuitry is 
well known in the art. In one embodiment, the micro- 
processor required to execute the program may be a 
dedicated microprocessor. However, because the 
global motion estimation computation is relatively 
simple and occurs only once per video frame, the pro- 
gram may be executed by a shared microprocessor, 
including, for example, one used for timing and con- 
trol of the overall coder circuit 

Essentially, the global motion estimator gener- 
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ates a global motion vector for a video frame by esti- 
mating the component of motion common to every 
block in the entire frame. This motion component is 
typically caused by camera motion or panning. The 
global motion estimator according to the present in- 5 
vention uses the block motion vectors associated with 
the previous frame to calculate the present global mo- 
tion component 

A global motion estimator embodiment that exe- 
cutes the steps illustrated in FIG. 4 operates by first 10 
defining a common motion vector by taking the aver- 
age of the motion vectors stored during the coding of 
a previous frame. A new global motion vector may 
then be provided to a motion estimator programmed 
to operate as discussed above in reference to FIG. is 
3 A. Other methods for estimating the global motion 
may be readily employed by one of ordinary skill in 
the art. 

In step 401, motion vector data defining a motion 
vector, MV|,dt for a block i, between a frame and a ref- 20 
erence frame dt frames apart, Is received from a 
block-based motion estimator. The motion estimator 
may suitably operate in accordance with FIG. 3A 
above to effect block matching and motion vector 
generation. In step 401 , the received motion vector is 25 
also normalized so that it represents the motion be- 
tween consecutive frames. A normalized motion vec- 
tor is given by equation MVt,di/dt where dt is the cur- 
rent frame number minus the reference frame num- 
ber. In step 402, the motion vector, MV|,dt. is added to 30 
a motion vector total, MVT, for the frame. Alternative- 
ly, the motion vector. MV},dt> niay be stored in a menri- 
ory in order to facilitate alternative global motion vec- 
tor computation methods, as discussed further be- 
low. In step 403, a counter C is advanced to track the 35 
number of motion vectors received for the current 
frame. 

In step 404, the global motion estimator may re- 
turn to receive another motion vector for the current 
block i. If the coder circuit employs motion compen- 40 
sation interpolation, multiple motion vectors, typically 
two, are generated during the processing of each new 
block of video data. Control circuitry typically pro- 
vkies this information to the global motion estimator 
so that the appropriate decision in step 404 can be 45 
made. If there are no more motion vectors for the 
block i, step 405 is executed. 

In step 405, the global motion estimator determi- 
nes whether there is another block of the current vid- 
eo frame to be motion compensated, in other words, so 
whether another motion vector Is expected for the 
frame Ft. If another motion vector is expected for the 
current frame, the program returns to step 401 and 
waits for the next motion vector. MV|,dt from the mo- 
tion estimation device. If. however, another motion 55 
vector is not expected for the frame, the global motion 
estimator executes step 406. 

In step 406 the common motion vector is caicu- 
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lated for the current frame. The common motion vec- 
tor may be the average motion vector determined by 
dividing the motion vector total by the number of vec- 
tors, C, received for the frame, or in other words, 
MVT/C. Alternatively, the common motion vector may 
suitably be the median motion vector or some other 
value as discussed below. 

In step 407, the global motion vector GMV is gen- 
erated for use in motion estimating the next frame, 
which for motion compensation prediction would be 
the frame F^-i. Because the motion vectors are nor- 
malized as they are received, the resulting global mo- 
tion vector will also already be normalzed. As dis- 
cussed above, the global motion vector may then be 
scaled in the motion estimator to allow for block 
matching therein with respect to any reference frame. 
The global motion vector GMV may suitably be the 
common motion vector calculated In step 406. It 
should be noted, however, that the use of multiple 
common motion vectors to generate a global motion 
vector may allow more sophisticated methods of es- 
timating the global motion. 

In step 408, the global motion vector is provided 
to a connection to the motion estimator which is op- 
erable to perform block-based motion compensation. 
The motion estimator 104 may suitably perform the 
method discussed above in connection with FIG. 3A. 

In the functional block 409, the counter C and mo- 
tion vector total MVT of the global motion estimator 
are reinitialized to prepare for the global motion vector 
calculation for the next video frame. 

In an alternative embodiment of the global motion 
estimation method, the median motion vector from a 
plurality of stored motion vectors may provide the 
common motion vector in step 406. In such an em- 
bodiment step 402 of FIG. 4 would include the oper- 
atk)n of storing the motion vector MVi^^jt- S^^p 406 
would then include the step of analyzing the several 
stored motion vectore to determine the median. The. 
foregoing steps may be readily implemented. 

in yet another alternative embodiment the global 
motion vector may suitably be determined from a his- 
togram. Under this method, the motion vector occur- 
ring with the most repetition within the processing of 
a frame would constitute the global motion vector. 

To implement this embodiment step 402 would 
include the step of updating a histogram of a table of 
possible motion vector values. For example, if MVi^^t 
= (0,+3), the occurrence index for (0,+3) within the ta- 
ble of possible motion vector values is incremented by 
one. The population of possible motion vectors may 
suitably be created for each new frame, whereby the 
value of motion vector MV|,dt is added to the popula- 
tion unless it is already represented. For example, if 
MV|,dt = (0,'*-3), and no occurrence index for that value 
exists, then (0,'i'3) is added and its index is increment- 
ed to one. 

Additionally, step 405 would include the step of 
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scanning the table for the largest occurrence index. 
The motion vector value occurring the most is then 
determined to be the common motion vector. The pro- 
giamming necessarily to achieve the foregoing may 
be readily accomplished. 

The usefulness of this embodiment is clear. It 
may assumed that most objects (and blocks) within a 
video frame do not exhibit independent movement. 
For example, background objects in most scenes are 
relatively static. However, if camera motion or global 
motion is present, these otherwise static objects will 
ail have the same motion vector. Because indepen- 
dently moving objects having the exact same motion 
vector will seldom outnumber the static objects, the 
motion vector having the most occurrences within a 
video frame is assumed to represent the global mo- 
tion. 

It is to be understood that the above-described 
arrangements of the invention are merely illustrative. 
Other arrangements may be devised by those skilled 
in the art which will embody the principles of the in- 
vention and fall within the spirit and scope thereof. 



Claims 

1. A method of converting a block of video data 
which is one of a plurality of blocks defining a 
frame of video data, into a compressed encoded 
digital bit stream for transmission over a trans- 
mission medium, said method comprising: 

a) generating a global motion vector; 

b) generating both video data and position 
data corresponding to the block of video data 
to be coded; and 

c) effecting block matching by using predeter- 
mined criteria to identify a displaced block 
corresponding to the block of video data to be 
coded, the displaced block having a location 
within a reference frame, said block matching 
step further comprising 

defining a search window within the reference 
frame using the global motion vector. 

2. The method of claim 1 further comprising the 
steps: 

d) generating a motion vector defined by the 
block position data and the location of the dis- 
placed block within the reference frame; 

e) storing said motion vector; 

f) encoding said motion vector into a bit 
stream for transmission; and 

g) repeating steps b) through 0 for a plurality 
of blocks to be encoded, said plurality of 
blocks constituting at least a portion of one 
frame of video data. 



erating the global motion vector further compris- 
es the step of utilizing a plurality of the previously 
stored motion vectors from said step (e) to gen- 
erate the global motion vector and further com- 
5 prising the step of: 

h) repeating steps a) through g) for a plur- 
ality of frames of video data. 

4. The method of claim 3 wherein the plurality of 
10 previously stored motion vectora comprise mo- 
tion vectora stored during the encoding of a frame 
of video data. 

5. The method of claim 3 wherein the step of gen- 
15 erating the global motion vector utilizing a plural- 
ity of the previously stored motion vectors further 
comprises utilizing the mean of the plurality of the 
previously stored motion vectors to generate the 
global motion vector. 

20 

6- The method of claim 3 wherein the step of gen- 
erating the global motion vector utilizing a plural- 
ity of the previously stored motion vectors further 
comprises utilizing the median of the plurality of 
25 the previously stored motion vectora to generate 

the global motion vector. 

7. The method of claim 3 wherein the step of gen- 
erating the global motion vector utilizing a plural- 

30 ity of the previously stored motion vectors further 

comprises utilizing the motion vector occurring 
with the most repetition within the plurality of the 
previously stored motk>n vectors to generate the 
global motion vector. 

35 

8. An apparatus for determining the location of a 
displaced block of video data within a reference 
frame corresponding to a block of video data 
where the displaced block Is identified from a 

40 plurality of candidate displaced blocks using pre- 

determined criteria, the apparatus comprising: 

a) a motton estimator comprising a process- 
ing means and memory means, the process- 
ing means operable to determine the location 

45 of a displaced block by defining a search win- 

dow within the reference frame using a global 
motion vector. Identifying a plurality of candi- 
date displaced blocks having a location with 
the search window, and comparing each can- 
so didate displaced blocks with the block of video 
data; and 

b) a global motion estimator coupled to the 
motion estimator, comprising processing 
means and storage means, said processing 

55 means operable to generate global motion 

vectors. 



3. The method of daim 2 wherein the step of gen- 9. The apparatus of daim 8 wherein the motksn es- 
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10. 



timator further comprises means for effecting 
motion compensation prediction. 

The apparatus of daim 8 wherein the motion es- 
timator further comprises means for effecting 
motion compensation interpolation. 



11. A video coder circuit for converting a plurality of 
frames of video data comprising a plurality of 
pels, each pel having pel video data, a plurality of 
pels further defining a block, into a compressed 
encoded digital bit stream for transmission over 
a transmission medium, said video coder circuit 
comprising: 

a) a video frame storage unit operable to gen- 
erate a block of video to be coded from a 
frame of video data, said block having a pos- 
ition within the frame of video data; 

b) a motion estimation means connected to 
the video frame storage unit for receiving the 
video data corresponding to a block of video 
to be coded, said motion estimation means 
operable to effect block matching between a 
block of video to be coded and a block within 
a reference frame to identify a displaced 
block having a location within the reference 
frame, and to generate a motion vector de- 
fined by the position of the block to be coded 
and the location of the displaced block within 
the reference frame, said motion estimation 
means further operable to effect block match- 
ing using a global motion vector; 

c) a first buffer for storing video data corre- 
sponding to one or more reference frames 
connected to the motion estimation means for 
providing reference frame video data thereto; 

d) a second buffer for storing video data cor- 
responding to a current frame connected to 
the first buffer for providing current frame vid- 
eo data thereto and connected to the motion 
estimation means; 

e) a subtraction node connected to the video 
frame storage unit for receiving block video 
data and connected to the motion estimation 
means for receiving displaced block video 
data, said subtraction node operable to gen- 
erate an error signal representative of the dif- 
ference in pel video data between two blocks 
of video data; and 

f) a coding device connected to the subtrac- 
tion node for receiving an error signal and con- 
nected to the motion estimation means for re- 
ceiving a motion vector, said coding device 
operable to encode video data from a block 
error signal and motion vector data into a bit 
stream. 

12. The system of claim 11 further comprising a glo- 



bal motton estimation means connected to the 
motion estimation means for providing the global 
motion vector thereto, said global motion estima- 
tion means operable to generate global motion 
5 vector data representative of the global motion 

between video frames. 

13. The system of claim 12 wherein the global motion 
estimation means is operably connected to re- 
10 ceive motion vectors from the motion estimation 

means and is further operable to generate a glo- 
bal motion vector from a plurality of received mo- 
tion vectors. 

15 14. The video coder circuit of claim 11 wherein the 
motion estimation means further comprises 
means for effecting motbn compensation predic- 
tion. 

20 15. The video coder circuit of claim 11 wherein the 
motion estimation means further comprises 
means for effecting motion compensation inter- 
polation. 

25 16. A method of defining a search window within a 
stored reference video frame for use in block 
matching in a block-based motion estimation de- 
vice, the block-based motion estimation device 
being operable to effect block matching and gen- 

30 erate motion vectors representing the displace- 

ment between a new video data block and a dis- 
placed block within the reference video frame, 
the search window definition method comprising: 

a) generating a motion vector for the new data 
35 block from a frame of video data comprising 

a plurality of data blocks; 

b) storing said motion vector in a memory; 

c) repeating steps a) and b) until a plurality of 
motion vectors for a plurality of said data 

40 blocks within said frame of video data have 

been generated and stored; 

d) generating a global motion vector from the 
plurality of the stored motion vectors; 

e) identifying a video data block having a lo- 
45 cation within a frame of video data to be cod- 
ed; and 

f) defining a search window having a location 
within a stored reference video frame wherein 
the location of the search window is at least 

50 partially dependent upon the global motion 

vector 

17. The method of claim 16 wherein the location of 
the search window to be defined in step f) is at 
55 least partially dependent on the location of the 

video data block within the frame of video data to 
be coded. 
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18. The method of daim 16 wherein the step of gen- 
erating the giobal motion vector using a predeter- 
mined method further comprises the step of cal- 
culating the mean of the plurality of stored motion 
vectors. 5 

19- The method of claim 16 wherein the step of gen- 
erating the global motion vector using a predeter- 
mined method further comprises the step of de- 
termining the median of the plurality of stored mo- io 
tlon vectors. 

20. The method of daim 16 wherein the step of gen- 
erating the global motion vector using a predeter- 
mined method further comprises the step of de- is 
termining the motion vector occurring with the 
most repetition within the plurality of stored mo- 
tion vectors. 

21. The method of daim 20 wherein the step of gen- 20 
erating the global motion further comprises the 
step of creating a table of occurrence indices for 

a plurality of possible motion vectors and the step 
of storing the motion vector comprises increment- 
ing an occurrence Index corresponding to the mo- 25 
tion vector in the table. 
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@ Method and apparatus for improving motion compensation in digital video coding. 



@ A novel method and apparatus for use in 
digital video compression provides improved 
block-based motion compensation using the 
global motion of a video frame. A video frame 
comprising a plurality of blocks may be com- 
pressed for transmisston using btock mctk^n 
vectors. Motion vectors are generated by block 
matching a block to be coded wrtti a block 
within a reference frame and determining the 
displacement therebetween. To effect block 
matching, motion compensation techniques de- 
fine a search window within one or more refer- 
ence frames within which a displaced block will 
be found. According to the present invention, 
the location of the search window within a 
reference frame is defined using the global 
motion of the frame. 
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